Evolutionary Games for Multiple Access Control*"'' 



Quanyan Zhu, Hamidou Tembine, Tamer Ba§ar-'- 
January 14, 2013 

^ ! 

, Abstract 

^ , In this paper, we formulate an evolutionary multiple access control game with continuous- 

, ^ ' variable actions and coupled constraints. We characterize equilibria of the game and show that 

the pure equilibria are Pareto optimal and also resilient to deviations by coalitions of any size, 
i.e., they are strong equilibria. We use the concepts of price of anarchy and strong price of 
anarchy to study the performance of the system. The paper also addresses how to select one 
specific equilibrium solution using the concepts of normalized equilibrium and evolutionarily 
stable strategies. We examine the long-run behavior of these strategies under several classes 
, of evolutionary game dynamics, such as Brown-von Neumann-Nash dynamics. Smith dynamics 

\^ , and replicator dynamics. In addition, we examine correlated equilibrium for the single-receiver 

C/3 ' model. Correlated strategies are based on signaling structures before making decisions on rates. 

^ , I We then focus on evolutionary games for hybrid additive white Gaussian noise multiple access 

channel with multiple users and multiple receivers, where each user chooses a rate and splits it 
over the receivers. Users have coupled constraints determined by the capacity regions. Building 
^ ' upon the static game, we formulate a system of hybrid evolutionary game dynamics using G- 

function dynamics and Smith dynamics on rate control and channel selection, respectively. We 
show that the evolving game has an equilibrium and illustrate these dynamics with numerical 
(SJ ! examples. 

rn 

5 ! 1 Introduction 



X 



Recently, there has been much interest in understanding the behavior of multiple access controls 
under constraints. Considerable amount of work has been carried out on the problem of how users 
can obtain an acceptable throughput by choosing rates independently. Motivated by the interest 
. in studying a large population of users playing the game over time, evolutionary game theory was 

found to be an appropriate framework for communication networks. It has been applied to problems 
such as power control in wireless networks and mobile interference control |1H fH [5| [6], 

The game-theoretical models considered in the previous studies on user behaviors in CDMA, 
[37\ H], are static one-shot non-cooperative games in which users are assumed to be rational and 
optimize their payoffs independently. Evolutionary game theory, on the other hand, studies games 
that are played repeatedly, and focuses on the strategies that persist over time, yielding the best 
fitness of a user in a non-cooperative environment on a large time scale. 

In [19] , an additive white Gaussian noise ( AWGN) multiple access channel problem was modeled 
as a noncooperative game with pairwise interactions, in which users were modeled as rational 
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entities whose only interest was to maximize their own communication rates. The authors obtained 
the Nash equiUbrium of the two-user game and introduced a two-player evolutionary game model 
with pairwise interactions based on replicator dynamics. However, the case when interactions are 
not pairwise arises frequently in communication networks, such as the Code Division Multiple 
Access (CDMA) or the Orthogonal Frequency-Division Multiple Access (OFDMA) in Worldwide 
Interoperability for Microwave Access (WiMAX) environment [11]. 

In this work, we extend the study of [19] to wireless communication systems with an arbitrary 
number of users corresponding to each receiver. We formulate a static non-cooperative game 
with m users subject to rate capacity constraints, and extend the constrained game to a dynamic 
evolutionary game with a large number of users whose strategies evolve over time. Different from 
evolutionary games with discrete and finite number of actions, our model is based on a class of 
continuous games, known as continuous-trait games. Evolutionary games with continuum action 
spaces can be encountered in a wide variety of applications in evolutionary ecology, such as evolution 
of phenology, germination, nutrient foraging in plants, and predator-prey foraging [24:\ [7] . 

In addition to the single receiver model, we investigate the case with multiple users and receivers. 
We first formulate a static hybrid non-cooperative game with users who rationally make decisions 
on the rates as well as the channel selection subject to rate capacity constraints of each receiver. 
We extend the static game to a dynamic evolutionary game by viewing rate selections governed by 
a fitness function parameterized by the channel selections. Such a concept of a hybrid model has 
appeared earlier in |36) and [IQ], in the context of hybrid power control in CDMA systems. The 
strategies of channel selections determine the long-term fitness of the rates chosen by each user. 
We formulate such dynamics based on generalized Smith dynamics and generating fitness function 
(G-function) dynamics. 

1.1 Contribution 

The main contributions of this work can be summarized as follows. We first introduce a game- 
theoretic framework for local interactions between many users and a single receiver. We show that 
the static continuous-kernel rate allocation game with coupled rate constraints has a convex set of 
pure Nash equilibria, coinciding with the maximal face of the polyhedral capacity region. All the 
pure equilibria are Pareto optimal and are also strong equilibria, resilient to simultaneous devia- 
tion by coalition of any size. We show that the pure Nash equilibria in the rate allocation problem 
are 100% efficient in terms of Price of Anarchy (PoA) and constrained Strong Price of Anarchy 
(CSPoA). We study the stability of strong equilibria, normalized equilibria, and evolutionary stable 
strategies (ESS) using evolutionary game dynamics such as Brown-von Neumann-Nash dynamics, 
generalized Smith dynamics, and replicator dynamics. We further investigate the correlated equi- 
librium of the multiple access game where the receiver can send signals to the users to mediate the 
behaviors of the transmitters. 

Based on the single-receiver model, we then propose an evolutionary game-theoretic framework 
for the hybrid additive white Gaussian noise multiple access channel. We consider a communication 
system of multiple users and multiple receivers, where each user chooses a rate and splits it over the 
receivers. Users have coupled constraints determined by the capacity regions. We characterize Nash 
equilibrium of the static game and show the existence of the equilibrium under general conditions. 
Building upon the static game, we formulate a system of hybrid evolutionary game dynamics using 
G-function dynamics and Smith dynamics on rate control and channel selection, respectively. We 
show that the evolving game has an equilibrium and illustrate these dynamics with numerical 
examples. 
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Table 1: List of Notations 



Symbol 


Meaning 


A/" 


bet oi iV ubeib 


o 


a subset of IV users 


J 


beu Oi J leceiveio 


A 

•^i 


action set of user i 


JD 

-Tj 


maximum power of user i 


hi 


channel gain of user i 


Oil 


rate of user i 


Pij 


probability of user i selecting receiver j 


Ui 


payoff of user i 




expected payoff of user i 


c 


capacity region of a set J\f of users in a single receiver case 


C{j) 


capacity region of a set A/" of users at receiver j 




distribution over the action set Ai 




population state 



1.2 Organization of the paper 

The rest of the paper is structured as follows. We present in Section 12.11 the evolutionary game 
model of rate allocation in additive white Gaussian multiple access wireless networks, and analyze 
its equilibria and Pareto optimality in Section 12.21 In Section 12.31 we present strong equilibria 
and price of anarchy of the game. In Section 12.41 we discuss how to select one specific equilibrium 
such as normalized equilibrium and evolutionary stable strategies. Section [2.51 studies the stability 
of equilibria and evolution of strategies using game dynamics. Section 12.61 analyzes the correlated 
equilibrium of the multiple acess game. 

In Section 13. H we present the hybrid rate control model where users can choose the rates and 
the probability of the channels to use. In Section 13.21 we characterize the Nash equilibrium of the 
constrained hybrid rate control game model, pointing out the existence of the Nash equilibrium of 
the hybrid model and methods to find it. In Section [3.3[ we apply evolutionary dynamics to both 
rates and channel selection probabilities. We use simulations to demonstrate the validity of these 
proposed dynamics and illustrate the evolution of the overall evolutionary dynamics of the hybrid 
model. Section U] concludes the paper. For reader's convenience, we summarize the notations in 
Table [D and the acronyms in Table [21 

2 AWGN Mutiple Access Model: Single Receiver Case 

We consider a communication system consisting of several receivers and several senders (see Figure 
[T]). At each time, there are several simultaneous local interactions (typically, at each receiver there 
is a local interaction). Each local interaction corresponds to a non-cooperative one-shot game with 
common constraints. The opponents do not necessarily stay the same from a given time slot to the 
next one. Users revise their rates in view of their payoffs and the coupled constraints (for example 
by using an evolutionary process, a learning process or a trial-and-error updating process). The 
game evolves over time. Users are interested in maximizing a fitness function based on their own 
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Table 2: List of Acronyms 



Abbreviation 


Meaning 


^ V J VV 1> 


Tt^LIU-I Ijl Vc V V 111 Lc vJcxLloolclll i > Uloc 


MAC 


Multiple Access Control 


MISO 


Multi-Input and Multi-Output 


CCE 


Constrained Correlated Equilibrium 


ESS 


Evolutionary Stable Equilibrium 


NE 


Nash Equilibrium 


PoA 


Price of Anarchy 


SPoA 


Strong Price of Anarchy 



communication rates at each time, and they are aware of the fact that the other users have the 
same goal. The coupled power and rate constraints are also common knowledge. Users have to 
choose independently their own coding rates at the beginning of the communication, where the 
rates selected by a user may be either deterministic, or chosen from some distribution. If the rate 
profile arrived at as a result of these independent decisions lies in the capacity region, users will 
communicate at that operating point. Otherwise, either the receiver is unable to decode any signal 
and the observed rates are zero, or only one of the signals can be decoded. The latter occurs 
when all the other users are transmitting at or below a safe rate. With these assumptions, we can 
define a constrained non-cooperative game. The set of allowed strategies for user i is the set of 
all probability distributions over [0, +oo[, and the payoff is a function of the rates. In addition, 
the rational action (rate) sets are restricted to lie in the capacity regions (the payoff is zero if 
the constraint is violated). In order to study the interactions between the selfish or partially 
cooperative users and their stationary rates in the long run, we propose to model the problem of 
rate allocation in Gaussian multiple access channels as an evolutionary game with a continuous 
action space and coupled constraints. The development of evolutionary game theory is a major 
contribution of biology to competitive decision making and the evolution of cooperation. The key 
concepts of evolutionary game theory are (i) Evolutionary Stable States [27], which is a refinement of 
equilibria, and (ii) Evolutionary Game Dynamics such as replicator dynamics [32], which describes 
the evolution of strategies or frequencies of use of strategies in time, [71 |2T|. 

The single population evolutionary rate allocation game is described as follows: there is one 
population of senders (users) and several receivers. The number of senders is large. At each time, 
there are many one-shot games called local interactions. Each sender of the population chooses 
from his set of strategies Ai which is a non-empty, convex and compact subset of M. Without loss 
of generality, we can suppose that user i chooses its rate in the interval Ai = [0, C|j}], where C{j} is 
the rate upper bound for user i (to be made precise shortly), as outside of the capacity region the 
payoff (as to be defined later) will be zero. Let A{Ai) be the set of probability distributions over 
the pure strategy set Ai. The set A.{Ai) can be interpreted as the set of mixed strategies. It is also 
interpreted as the set of distributions of strategies among the population. Let Aj G A{Ai), and E 
be a Aj— measurable subset of M^; then Xi{E) represents the fraction of users choosing a strategy 
out of E, at time t. A distribution Aj G A(^j) is sometimes called the "state" of the population. 
We denote by M{Ai) the Borel ct— algebra on Ai and by d{X,X') the distance between two states 
measured with the respect to the weak topology. Each user's payoff depends on opponents' behavior 
through the distribution of opponents' choices and of their strategies. The payoff of a user z in a 
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Figure 1: A population: distributed receivers and senders, represented by blue rectangles and red 
circles respectively. 



local interaction with (A^ — 1) other users is given as a function Ui : M — )■ M. The rate profile 
a € must belong to a common capacity region C C defined by 2^ — 1 linear inequalities. 
The expected payoff of a sender i transmitting at a rate a when the state of the population is 
G A(^j) is given by Fi{a,n). The expected payoff for user i is 




The population state is subjected to the "mixed extension" of capacity constraints A4{C). This will 
be discussed in Section [231 and will be made more precise later. 



2.1 Local Interactions 

Local interaction refers to the problem setting of one receiver and its uplink additive white Gaussian 
noise (AWGN) multiple access channel with A^ senders with coupled constraints (or actions). The 
signal at the receiver is given by y = + X^^^ Xi where Xi is a transmitted signal of user i and 
^ is a zero- mean Gaussian noise with variance ctq. Each user has an individual power constraint 
IE (A?) < Pi and the channel gain hi. The optimal power allocation scheme is to transmit at the 
maximum power available, i.e. Pi, for each user. Hence, we consider the case in which maximum 
power is attained. The decisions of the users then consist of choosing their communication rates, 
and the receiver's role is to decode, if possible. The capacity region is the set of all vectors a G 
such that users i & J\f := {1,2, . . . , A^} can reliably communicate at rate Oj, i G Af. The capacity 
region C for this channel is the set 



VQi<log + .V0CJ7CAr' 



Example 1. (Example of capacity region with three users) In this example, we illustrate the capacity 
region with three users. Let ai,a2,«3 be the rates of the users and Pi = P,hi = h,\/i = 1,2,3. 
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Based on ([ip, we obtain a set of inequalities 

' ai > 0,Q2 > 0,03 > 



ai<los(l + ^),i = 1,2,3 



a* + a, < log (1 + 2^^) = 1,2,3. ' 



«! + 02 + 03 < log ( 1 + 3 



or in the compact notation, -^373 < (^3, where 



iPh 



73 : = 



"3 



-) C3 := 



C{i} 




" 1 





" 


C{2} 







1 





C{3} 










1 


C{1,2} 


,M3 := 


1 


1 





C{1,3} 




1 





1 


C{2,3} 







1 


1 


- ^1,2,3} - 




1 


1 


1 _ 



€ Z 



7x3 



Note that M3 is a totally unimodular matrix. By letting Ph = 25,(Tq = 0.1, we sketch in Figure\^ 
the capacity region with three users. 

The capacity region reveals a competitive nature of the interactions among senders: if a user i 
wants to communicate at a higher rate, one of the other users has to lower his rate; otherwise, the 
capacity constraint is violated. We let 



:= log 1 + 



Pihi 



, i G A/", C TV 



denote the bound on the rate of a user when the signals of the |r2| — 1 other users are treated as 
noise. 

Due to the noncooperative nature of the rate allocation, we can formulate the one-shot game 

H = (A/", (A)ieA^, {ui)i(^M) , 

where the set of users is the set of players, Ai, i G M, is the set of actions, and Ui, i ^ M, are 
the payoff functions. 



2.2 Payoffs 

We define Ui : Hi^i -^i 



M-l- as follows: 
Ui{ai,a-i) = 



gi{ai) if (ai,a_i) G C 
otherwise 



(1) 
(2) 



where ]c is the indicator function; a_i is a vector consisting of other players' rates, i.e., a_i = 
[ai, . . . , aj-i, Oj+i, . . . , un] and Ui is a positive and strictly increasing function for each fixed a_j. 
Since the game is subject to coupled constraints, the action set Ai is coupled and dependent on 
other players' actions. Given the strategy profile a_i of other players, the constrained action set 
Ai is given by 

Ai{a^i) := {ai £ [0, C{i}], (Qj,a_i) e C} (3) 
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Figure 2: Capacity region with three users. 



We then have an asymmetric game. The minimum rate that the user i can guarantee in the feasible 
regions is ri^j\f which is different than rj^j^/. 

Each user i maximizes Ui(ai, over the coupled constraint set. Owing to the monotonicity of 
the function gi and the inequalities that define the capacity region, we obtain the following lemma. 

Lemma 1. Let BRi[a-i) he the best reply to the strategy a-i, defined by 



BRi is a non-empty single-valued correspondence (i.e. a standard function), and is given by 



BR (a-i) = arg max Ui{y,a-i). 

y(iAi{a-i) 




where Fj = {$7 G 2- 



i G 17}. 
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Proposition 1. The set of Nash equilibria is 
All these equilibria are optimal in Pareto sense. 

Proof. Let /3 be a feasible solution, i.e., /3 G C. If f3i < Cf^ = log ^1 + Yli&M '^^) ' then at 
least one of the users can improve its rate (hence its payoff) to reach one of the faces of the capacity 
region. We now check the strategy profile on the face 

N \ 



i=l 



Oil > fi,N'^Y^f=iOii = Coj- , then from the Lemma [H BRi{(3^i) = {Pi}. Hence, 

arium. Moreover, this strategy j3 is Pareto optimal because the rate of each user 
is maximized under the capacity constraint. These strategies are social welfare optimal if the total 
utility Yld=i'^i{^i^'^-i) — YliLidii^i) maximized subject to constraints. □ 



If /3 G |(ai,a_i) 
/3 is a strict equili 



Note that the set of pure Nash equilibria is a convex subset of the capacity region. 

2.3 Robust Equilibria and Efficiency Measures 

2.3.1 Constrained Strong Equilibria and Coalition Proofness 

An action profile in a local interaction between senders is a constrained A;— strong equilibrium if 
it is feasible and no coalition of size k can improve the rate transmissions of each of its members 
with respect to the capacity constraints. An action is a constrained strong equilibrium [18] if it is a 
constrained A;— strong equilibrium for any size k. A strong equilibrium is then a policy from which 
no coalition (of any size) can deviate and improve the transmission rate of every member of the 
coalition (group of the simultaneous moves) , while possibly lowering the transmission rate of users 
outside the coalition group. This notion of constrained strong equilibris0 is very attractive because 
it is resilient against coalitions of users. Most of the games do not admit any strong equilibrium 
but in our case we will show that the multiple access channel game has several strong equilibria. 

Theorem 1. Any rate profile on the maximal face of the capacity region C : 

-Facemax(C) := |(ai,a_i) G | > rAr,^ai = Cvj , 
is a constrained strong equilibrium. 

Proof. We remark that if the rate profile a is not on the maximal face of the capacity region, 
then a is not resilient to deviation by a single user. Hence, a cannot be a constrained strong 
equilibrium. This says that a necessary condition for a rate profile to be a strong equilibrium is to 
be in the subset Facemax(C). We now prove that the condition: a G Facemax(C) is sufficient. Let 
a G Facemax(C). Suppose that k users deviate simultaneously from the rate profile a. Denote by 
Dev the set of users which deviate simultaneously (eventually by forming a coalition). The rate 
constraints of the deviants are 



^Note that the set of constrained strong equihbria is a subset of the set of Nash equihbria (by taking coalitions of 
size one) and any constrained strong equihbrium is Pareto optimal (by taking coalition of full size). 
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1. a'^ > 0, G Dev, 

2- T.^en<<C^^ yncDev, 

3- T.^ennDev < < Cn - T^^eUMDev C A/", J] n Dev / 0. 

In particular, for Q, = M, we have YlieDev'^'i — ^A/" ~ 'l2i^Dev'^i- '^^^ total rate of the deviants 
is bounded by CV — J2i^Dev '^i^ which is not controlled by the deviants. The deviants move to 
{o:'i)i(^Dev with 

^ a • < CV - ^ Oi. 

iGDev i^Dev 

Then, there exists i such that > a'^. Since gi is non-decreasing, this implies that gi{ai) > gi{ci[). 
The user i who is a member of the coalition Dev does not improve its payoff. If the rates of some 
of the deviants are increased, then the rates of some other users from coalition must decrease. If 
[a[)i(zDev satisfies 

iGDev i^Dev 

then some users in the coalition Dev have increased their rates compared with {ai)i^j:)ev aud some 
others in Dev have decreased their rates of transmission (because the total rate is the constant 
CV — Sj^Dev '^«)- "^^^ users in Dev with a lower rate a- < Oi do not benefit by being a member of 
the coalition (Shapley criterion of membership of coalition does not hold) . And this holds for any 
^ Dev ^ M. This completes the proof. □ 

Corollary 1. In the constrained rate allocation game, Nash equilibria and strong equilibria in pure 
strategies coincide. 

2.3.2 Constrained Potential Function for Local Interaction 

Introduce the following function: 

TV 



V{a) =]c (a) ^5i(ai) , 



i=l 



where ]c is the indicator function of C,i.e., ]c (a) = 1 if a € C and otherwise. The function V 
satisfies 

V{a) - V{(3i,a-i) = gi{ai) - g^i), Va, (/3i,a_i) G C. 



If gi is differentiable, then one has 



in the interior of the capacity region C, and y is a constrained potential function [3] in pure 
strategies. 

Corollary 2. The local maximizers ofV in C are pure Nash equilibria. Global maximizers ofV in 
C are both constrained strong equilibria and social optima for the local interaction. 
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2.3.3 Strong Price of Anarchy 

Throughout this subsection, we assume that the functions gi are the identity function, i.e., gi{x) = 
id{x) := X. One metric used to measure how much the performance of decentrahzed systems is 
affected by the selfish behavior of its components is the price of anarchy. We present a similar 
price for strong equilibria under the coupled rate constraints. This notion of Price of Anarchy can 
be seen as an efficiency metric that measures the price of selfishness or decentralization and has 
been extensively used in the context of congestion games or routing games where typically users 
have to minimize a cost function [411 B2] . In the context of rate allocation in the multiple access 
channel, we define an equivalent measure of price of anarchy for rate maximization problems. One 
of the advantages of a strong equilibrium is that it has the potential to reduce the distance between 
the optimal solution and the solution obtained as an outcome of selfish behavior, typically in the 
case where the capacity constraint is violated at each time. Since the constrained rate allocation 
game has strong equilibria, we can define the strong price of anarchy, introduced in as the 
ratio between the payoff of the worst constrained strong equilibrium and the social optimum value 
which CV- 

Theorem 2. The strong price of anarchy of the constrained rate allocation game is 1 for gi{x) = x. 

Note that for gi ^ id, the CSPoA can be less than one. However, the optimistic price of anarchy 
of the best constrained equilibrium, also called price of stability [13], is one for any function g^ i.e 
the efficiency of "best" equilibria is 100%. 



2.4 Selection of Pure Equilibria 

We have shown in the previous sections that our rate allocation game has a continuum of pure 
Nash equilibria and strong equilibria. We address now the problem of selecting one equilibrium 
which has certain desirable properties: the normalized pure Nash equilibrium, introduced in [29] : 
see also [31^ [20l 123] . We introduce the problem of constrained maximization faced by each user 
when the other rates are at the maximal face of the polytope C: 

max Ui{a) (5) 

a 

s.t. ai + . . . + ttN = Cjsf (6) 
for which the corresponding Lagrangian for user i is given by 

Li(a,C) = Ui{a) - Ci (YI""^ ~ ■ 
From Karush-Kuhn- Tucker optimality conditions, it follows that there exists ( G such that 

N 

9iioii) = Ci, '^Oii = CV- 

i=l 

For a fixed vector C with identical entries, define the normal form game r(^) with users, where 
actions are taken as rates and the payoffs given by L{a, C). A normalized equilibrium is an equilib- 
rium of the game r((^*) where C* is normalized into the form Q = ^, c > 0,rj > 0. We now have 
the following result due to Goodman |:20j which implies Rosen's condition on uniqueness for strict 
concave games. 
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Theorem 3. Let Ui he a smooth and strictly concave function in a^, each ui he convex in a_j, and 
there exist some C, such that the weighted non-negative sum of the payoffs Yl^=i d'^ii'^) ^■^ concave 
in a. Then, the matrix G{a,C) + G'^(ck,C) negative definite (which implies uniqueness) where 
G{a, Q is the Jacobian with respect to a of 

h{a,C) ■= [CiVini(a),C2V2ii2(a), • • • ,CAfVArnAr(Q)]'^ 

and G^ is the transpose of the matrix G. 

This now leads to the fohowing corollary for our problem. 

Corollary 3. If gi are non- decreasing strictly concave functions, then the rate allocation game has 
a unique normalized equilibrium which corresponds to an equilibrium of the normal form game with 
payoff L (a, (*) for some (*■ 



2.5 Stability and Dynamics 

In this subsection, we study the stability of equilibria and several classes of evolutionary game 
dynamics under a symmetric case, i.e.. Pi = P,hi = h,gi = g,Ai = A, Vi G M. We will drop 
subscript index i where appropriate. We show that the associated evolutionary game has a unique 
pure constrained evolutionary stable strategy. 

Proposition 2. The collection of rates a. = • • • , i i-C- the distribution of Dirac concen- 

trated on the rate is the unique symmetric pure Nash equilibrium. 

Proof. Since the constrained rate allocation game is symmetric, there exists a symmetric (pure or 
mixed) Nash equilibrium. If such an equilibrium exists in pure strategies, each user transmits with 
the same rate r*. It follows from Proposition [T] of Section 12.21 ^'^^ the bound r^v < that r* 
satisfies Nr* = Cj\f and r* is feasible. □ 

Since the set of feasible actions is convex, we can define convex combination of rates in the set 
of the feasible rates. For example, ea' + (1 — e)a is a feasible rate if a' and a are feasible. The 
symmetric rate profile (r, r, . . . , r) is feasible if and only if < r < r* = We say that the rate 
r is a constrained evolutionarily stable strategy (ESS) if it is feasible and for every mutant strategy 
mut ^ a there exists €mut > such that 

f r^:= e mut + (1 - e)r S C Ve G (0, emut) 
\ u{r,r^, . . . ,r^) > u{mut,r^, . . . ,r^) VeG(0, e^ut) 

Theorem 1. The pure strategy r* = ^0- is a constrained evolutionary stable strategy. 

Proof. Let mut < r* The rate e mut + (1 — e)r* is feasible implies that mut < r* (because r* 
is the maximum symmetric rate achievable). Since mut ^ r* , mut is strictly lower than r*. By 
monotonicity of the function g, one has u{r* , e mut + (1 — e)r*) > u{mut, e mut + (1 — e)r*), Ve. 
This completes the proof. □ 



2.5.1 Symmetric Mixed Strategies 

Define the mixed capacity region 7W(C) as the set of measures profile (/ii,/X2, • • • ,/UAf) such that 



'^^+ \ien J ien 
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Then, the payoff of the action a € satisfymg (a, A, . . . , A) G -MiC) can be defined as 



F{a,fj,)= u{a,b2,. . . ,bN) i^N~i{db) , (7) 

J[0,oo[^-i 

where Uk = ^ is the product measure on [0, oo['^. The constraint set becomes the set of proba- 
bihty measures on such that 

< E(/i) := I a, ii{da,) <^< Qi} • 
Lemma 2. The payoff can be obtained as follows: 



-^(a, =1 [0,CAA-{Af-l)E(M)] Xfi'Ca) X / Z^Ar-l(d&) =1 [0,CAA-{m-l)E{/i)] X g{a)uN-i{'Da) , 

Jb&Va 

where Va = {(62, • • • , &Af) I (a, 62, • • • , &Af) G C} . 

Proof. If the rate does not satisfy the capacity constraints, then the payoff is 0. Hence the rational 
rate for user i is lower than C^j}. Fix a rate a € [0, C{j}]. Let := Cq — a(5{igQ}. Then, a 
necessary condition to have a non-zero payoff is (62, • • • j b^) S Pa , where Va = {(^2) • • • , &Af) G 
^+"^ Eiec,*^i < ^ 2^}. Thus, we have 



F{a,n) = I ^ u{a,b2,. . . ,bN) J^N^iidb) 

g{a) UN-i{db) 



i+ , (a,6)eC 

][o,Cj^-{N-i)E{f,)] gia) X / VN^i{db) 

JbeVa 



□ 



2.5.2 Constrained Evolutionary Game Dynamics 



The class of evolutionary games in large population provides a simple framework for describing 
strategic interactions among large numbers of users. In this subsection we turn to modeling the 
behavior of the users who play them. Traditionally, predictions of behavior in game theory are 
based on some notion of equilibrium, typically Cournot equilibrium, Bertrand equilibrium, Nash 
equilibrium, Stackelberg solution, Wardrop equilibrium or some refinement thereof. These notions 
require the assumption of equilibrium knowledge, which posits that each user correctly anticipates 
how his opponents will act. The equilibrium knowledge assumption is too strong and is difficult to 
justify in particular in contexts with large numbers of users. As an alternative to the equilibrium 
approach, we propose an explicitly dynamic updating choice, a procedure in which users myopically 
update their behavior in response to their current strategic environment. This dynamic procedure 
does not assume the automatic coordination of users' actions and beliefs, and it can derive many 
specifications of users' choice procedures. These procedures are specified formally by defining a 
revision of rates called revision protocol [30j . A revision protocol takes current payoffs and current 
mean rate and maps to conditional switch rates which describe how frequently users in some class 
playing rate a who are considering switching rates switch to strategy a' . Revision protocols are 
flexible enough to incorporate a wide variety of paradigms, including ones based on imitation, 
adaptation, learning, optimization, etc. 
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We use here a class of continuous evolutionary dynamics. We refer to [SSllIllIM] for evolutionary 
game dynamics with or without time delays. The continuous-time evolutionary game dynamics on 
the measure space {A, 13(A), fi) is given by 



Xt{E) = / Via,Xt)fiida) (8) 

Ja£E 



where 



Via,Xt) = K 



!3:{Xt)Xt{dx) - / fil{Xt)Xt{dx) 
t&A JxeA 

and /3J represents the rate of mutation from x to a, and is a growth parameter. /3J(Af) = 
if {x,Xt) or (a, At) is not in the (mixed) capacity region, ii^ is a /i— measurable subset of A. At 
each time t, probability measure At satisfies j^XtiA) = 0. We examine the following classes of 

evolutionary game dynamics, namely, Brown-von Neumann-Nash dynamics. Smith dynamics and 
replicator dynamics, where F{a, At) is the payoff in ([7]) as defined in the previous subsection. 

RD-1: Constrained Brown-von Neumann-Nash dynamics. 

max(F(a,At)- J^F{x,Xt) dx,0) if (a. At), {x,Xt)eM{C), 
otherwise. 



/3:(A*) 



RD-2: Constrained Replicator Dynamics. 

max(F(a, At) - F{x, At), 0) if (a. At), (x. At) € M{C) 



RD-3: Constrained ^— Smith Dynamics. 



otherwise. 



^ I max(F(a,At) -F(x,At),0)^ if (a, At), (x. At) G 7W(C) 
''^ ' otherwise. ' " 



A common property that applies to all these dynamics is that the set of Nash equilibria is a subset 
of rest points (stationary points) of the evolutionary game dynamics. Here we extend the concepts 
of these dynamics to evolutionary games with a continuum action space and coupled constraints, 
and more than two-users interactions. The counterparts of these results in discrete action space 
can be found in [211 ED]- 

Theorem 2. Any Nash equilibrium of the game is a rest point of the following evolutionary game 
dynamics: constrained Brown-von Neumann-Nash, generalized Smith dynamics, and replicator dy- 
namics. In particular, the evolutionary stable strategies set is a subset of the rest points of these 
constrained evolutionary game dynamics. 

Proof. It is clear for pure equilibria by using the revision protocols (3 of these dynamics. Let A be 
an equilibrium. For any rate a in the support of A, /3" = if F{x, A) < F{a, A). Thus, if A is an 
equilibrium, the difference between the microscopic infiow and outflow is V{a, A) = 0, given that a 
is the support of the measure A. □ 

Let A be a finite Borel measure on [0, C|j}] with full support. Suppose g is continuous on [0, C{j}]. 
Then, A is a rest point of the BNN dynamics if and only if A is a symmetric Nash equilibrium. Note 
that the choice of topology is an important issue when defining dynamics convergence and stability 
of the dynamics. The most used in this area is the topology of the weak convergence to measure 
closeness of two states of the system. Different distances (Prohorov metric, metric on bounded and 
Lipschitz continuous functions on A) have been proposed. We refer the reader to |26j, and the 
references therein for more details on evolutionary robust strategy and stability notions. 



13 



2.6 Correlated Equilibrium 

In this subsection, we analyze constrained correlated equilibria of multiple access (MAC) games. 
Building upon the signaling in the one-shot game, we formulate a system of evolutionary MAC 
games with evolutionary evolutionary game dynamics that describe the evolution of signaling, 
beliefs, rate control and channel selection, respectively. 

We focus on correlated equilibrium in the single-receiver case. Correlated strategies are based 
on signaling structures before making decisions on rates. Different scenarios (with or without 
mediator, virtual mediator, cryptographic multi-stage signaling structure) have been proposed in 
the literature [El [391 [El HZ] . 




Figure 3: Signaling between multiple senders and a receiver 

In Figure [3l we illustrate the signaling between multiple transmitters and one receiver. The 
receiver can act as a signaling device to mediate the behaviors of the transmitters. The correlated 
equilibrium has a strong connection with cryptography in that the private signal sent to the users 
can be realized by the coding and decoding in the network |39| . 

Let B be the set of signals /3 = [/3j,/3_j] G M^. The values (3 from the set of signals need to be 
in the feasible set C C M'^. Let /i € AB be a probability measure over the set B. A constrained 
correlated equilibrium (CCE) fi* need to satisfy the following set of inequalities, 

J d^*{(3i,(3^i) [ui{ai,a-i,\ - Ui{ai,a-i \ f3i)] > 0, Vi G AA, a- G A(a-j)- 

Define a rule of assignment of user i as a map its signals to its action's set fj : /3j i — > a^. A CCE 
is then characterized by 

J dn*{f3) [ui{ai,a-i \ j3i) - Mj(rj(/?i), a_j)] > 0, Vi G AA, Vrj such that fi(.) G A(a-i)- (9) 

Theorem 3. The set of constrained pure Nash equilibria of the MISO game is given by 

max-face{C) = < (ai, . . . , ajy) | ai > 0, Uk = > 
I keAf ) 

We can characterize the CCE using the above results as follows. 

Lemma 3. Any mixture of constrained pure Nash equilibria of the MISO game is a constrained 
correlated equilibrium. 
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Note that the set of constrained correlated equihbria is bigger than the set of constrained Nash 
equilibria. For example, in a two-user case, the distribution ^^{ri,c^i 2}-ri) + h^ic^i 2}-r2,r2) 
different than the Dirac distribution 5 ri+C|i2}~''2 '■2+C{i,2}-'i ■ 

Proof. Let /i be a probability distribution over some constrained pure equilibria. Then, fi G 
A(max-face(C)). For any /? such that /i(/3) 7^ 0, one has 

[ui{ai, a-i I f3i) - Ui{ri{f3i),a-i)] > 

for any measurable function fj : [0, C{j}] — > [0, C{j}]. Thus, /i is a constrained correlated equilib- 
rium. 

□ 

Corollary 4. Any convex combination of extreme point of the convex compact set 
max-face{C) 



{a = (ai, . . . , un) I ai > ri, ^ Ofc = CV > 



is a constrained correlated equilibrium. Moreover any probability distribution over the maximal face 
of the capacity region max-face{C) is a correlated constrained equilibrium distribution. 



3 Hybrid AWGN Multiple Access Control 

In this section, we extend the single receiver case to one with multiple receivers. Multi-input and 
multi-output (MIMO) channel access game has been studied in the context of power allocation 
and control. For instance, the authors in ^ formulate a two-player zero-sum game where the first 
player is the group of transmitters and the second one is the set of MIMO sub-channels. In [5], the 
authors formulate an A^-person non-cooperative power allocation game and study its equilibrium 
under two different decoding schemes. 

Here, we formulate a hybrid multiple access game where users are allowed to select their rates 
and channels under capacity constraints. We first obtain general results on the existence of the 
equilibrium and methods to characterize it. In addition, we investigate long-term behavior of the 
strategies and apply evolutionary game dynamics to both rates and channel selection probabilities. 
We show that G- function based dynamics is appropriate for our hybrid model by viewing the channel 
selection probabilities as strategies that determine of fitness of rate selection. Using the generalized 
Smith dynamics for channel selection, we are able to build an overall hybrid evolutionary dynamics 
for the static model. Based on simulations, we confirm the validity of these proposed dynamics and 
the correspondence between the rest point of the dynamics and the Nash equilibrium. 



3.1 Hybrid Model with Rate Control and Channel Selection 

In this subsection, we establish a model for multiple users and multiple receivers. Each user 
needs to decide the rate at which to transmit and the channel to pick. We formulate a game 
H = {Af,{Ai)i£j\f,{Ui)i^j\f), in which the decision variable is (aj,pj), and pi = \pij]j^j is a J- 
dimensional vector, where pij is the probability of user i & Af to choose channel j ^ J and pij 
needs to satisfy the probability measure constraints 

^Pii = l,Kj- >0,VzGAA (10) 

3^J 
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The game S is asymmetric in the sense that the strategy sets of users are different and the payoffs 
are not symmetric. 

Let Cj^Q := log ^1 + ^jgf^ ~^^^) capacity for a subset Q C J\f of users at receiver j £ J 

and rij Q := log ( 1 H — ^ '^'^ — r — ) the bound rate of a user i when the signals of the — 1 
other users are treated as noise at receiver j. Each receiver j has a capacity region C{j) given by 



C{j) = i{a,pj)e[0,lf X 



»iPiJ < Cj,Qy c % c A/-, j G i , (11) 



The expected payoff function Ui : YliLi — ^ ^+ of the game is given by 

Ui{ai,pi,a-i,p^i) = EpJuij(a,P)] = '^pijUij{a,P), (12) 

where a = (ai,a_i) G and P G [0,1]^^'^ = (pi,p_i), with G [0,1]-^, p_i G [0, Ij^^-^)^'^. 
Assume that the utility Uij of a user i transmitting to receiver j is only dependent on the user himself 
and is described by a positive and strictly increasing function gi : R_|_ — > M+, i.e., = Qi^j G J , 
when capacity constraints are satisfied. 

With the presence of coupled constraints (jlip from each receiver and probability measure con- 
straint pop , each sender has his individual optimization problem (lOP) given as follows. 

Pi 

Pij > 0,Vi G AA,j G J 
(a,p,-)GC(i),VjG J 

Denote the feasible set of (lOP) by J-" = J^i x J-'2, where 

J-i = {a G M^' I (a, P) G n,e^C(j), P G ^2} , (13) 

7-2 = |p G M^^^ I Y^pij = l,pij > 0, Vi G M,j G ^1 . (14) 

The action set of each user can thus be described by 

A(a-i,P-i) = {(ai,a_i) G Ti, (pi,p-i) G T2} . (15) 
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3.1.1 An Example 



Suppose we have three users and three receivers, that is, J\f = {1,2,3} and J7 
capacity region at receiver 1 is given by 



a, > 0, i = 1,2,3 



C(l) 



Piiai < iog(i + 



P2ia2 < log(l + 
Psias < log(l + 



Pnai+P2ia2 < log(l + AAiJ^) 
Piiai +P3ia3 < log(l + ^ihi^^) 
P2ia2 +P3ia3 < log(l + 
Piiai+P2ia2+P3ias < log(l + W^fma+ma) 
<Ki < 1, i = 1,2,3 



> . 



This can be written into 



C(l) 



Pi 



Pll 










Puai 


P21 


G[o,l]^ 


a2 




Ms 


P2ia2 


. P31 . 




. «3 . 






_ Psias _ 



< 



{1,2,3}. The 







Cl,{2} 




Cl,{3} 




^1,{1,2} 


> 


^1,{1,3} 




^1,{2,3} 




- C'i_{i^2,3} - 





where Ci^q, = log ( 1 + ^-g^^ 



and Ms is a totally unimodular matrix: Ms 
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1 

1 

1 1 

1 

1 1 
1 1 



Capacity regions at receivers 2 and 3 can be obtained in a similar way. 



3.2 Characterization of Constrained Nash Equihbria 

In this subsection, we characterize the Nash equilibria of the defined game H under the given 
capacity constraint. We use the following theorem to prove the existence of Nash equilibrium for 
the case where the rates are predetermined; this result is then used to solve the game for the case 
when both the rates and the connection probabilities are (joint) decision variables. 

Theorem 4. (Ba§ar & Olsder, 138^ ) Let A = Ai y~ A2 ■ ■ ■ x An be a closed, bounded and convex 
subset ofM.^, and for each i S M, the payoff functional Ui : A^M. be jointly continuous in A and 
concave in ai for every aj G Aj,j S J\f,j ^ i. Then, the associated N -person nonzero-sum game 
admits a Nash equilibrium in pure strategies. 

Applying Theorem [U we have the following results immediately. 
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Proposition 3. Suppose ai,i G A^, are predetermined feasible rates. Let feasible set T be closed, 
bounded and convex. If gi in (lOP) are continuous on J- and concave in pi (without the assumption 
of being positive and strictly increasing), the expected payoff functions Ui : M.^ x [0, 1]^^-^ — >• M are 
concave in Pi and continuous on T , then the static game admits a Nash equilibrium. 

The existence result in Proposition [3] only captures the case where the rates are predeter- 
mined, and relies on the convexity requirement of the utility functions g^. We can actually obtain 
a stronger existence result by observing that the formulated game H is a potential game with a 
potential function given by 

= ^fi{ai,Pi) = ^^Pij9i{aiPij), (16) 

i i j 

where = Yl,jPij9i^'^iPij)-> the expected payoff to user i. 

Note that the feasible set T is generally nonempty and bounded. We can conclude the existence 
result in Proposition [4] of NE from this observation. 

Proposition 4. The hybrid rate control game H admits a Nash equilibrium. 
Proof. Let us formulate a centralized optimization problem (COP) as follows. 

maXc^P ^'(a,P) 
s.t. (a,P) G 

Using the result in [3], we can conclude that if there exists a solution to (COP), then there exists 
a Nash equilibrium to the game H. Since J- is compact and nonempty, and the objective function 
is continuous, there exists a solution to (COP) and thus a Nash equilibrium to the game. □ 

The problem above is generally not convex and uniqueness of the Nash equilibrium may not be 
guaranteed. However, we still can further characterize the Nash equilibrium through the following 
propositions. 

Proposition 5. Let f3ij := aiPij. Without predetermining a, suppose that (p_i,Q_j) is feasible. A 
best response strategy at receiver j ^ J for user i must satisfy 

< PijCii < Cj^n^ - ^ akPkj,y^j (17) 

where ^Ij := {17 G 2-^ \ i' G ^,Pi'j > 0} is the set of users transmitting to receiver j. Since gi is a 
non- decreasing function, the best correspondence at j is 

P*o = o*p*j = max ^T-y.AT, min ^^sh ~ X! j ' (1^) 

where rij^j^ is the bound on the rate of user i when signals of \J\f \ — 1 other users are treated as 
noise. 

Proof. The proof is immediate by observing that the rate of user i at receiver j must satisfy l\17\i 
due to the coupled constraints. Thus, the maximum rate that user i can use to transmit to receiver 
j without violating the constraints is clearly the minimum of Cj^Uj — Yli'^i (^i'Pi'j o'^^r all Qj. Since 
the payoff is a non-decreasing function, the best response for i at receiver j is given by (jlSp . □ 
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Proposition 6. Let K* = argmaxj,zj gij{f3ij). If K* = {k*} is a singleton, then the best reply for 
user i is to choose 

( pij = 1 ifj = k*, 
\ Pij = otherwise, 

and we can determine ai by ai = 

Pik* 

If \Ki\ ^ 2, then the best response correspondence is 

( PieA{K*) ifjeK*, 
\ otherwise. 

We can determine ai from Pij by Oi = Yliji^K* l^ij- 
Proof. Since the expected utility is given in the form of 

Ui{ai,pi,a-i,p-i) = Ep^[uij{aiPij)], 

the expected utihty under best response is Ui = Kp-[uij{l3ij)]. If for a singleton k* such that 
k* = argmaxjg_/\/gjj(/3jj), we can assign all the weight pik* = 1 to maximize the expected utility. 
Since (3ik* = aiPik*, then a = (3ik*/pik* = (3ik*- If the set K* is not a singleton, without loss of 
generality, we can pick two indices £ K* such that /3jj = OiPij and /3jj' = OiPij', leading to 
Uij{j3ij) = Uiji{f3iji). Since the utilities to transmit using j and j' are the same, we can assign 
arbitrary (two-point) distribution, pjj and Pij' over them, with, pij +pij' = 1. Therefore, Pij + Pij' = 
ai{pij + pijf) = Oi. □ 

3.3 Multiple Access Evolutionary Games 

Interactions among users are dynamic and the users can update their rates and channel selection 
with respect to their payoffs and the known coupled constraints. Such a dynamic process can 
generally be modeled by either an evolutionary process, a learning process or a trial-and-error 
updating process. In classical game theory, the focus is on strategies that optimize payoffs to the 
players while, in evolutionary game theory, the focus is on strategies that will persist through time. 
In this subsection, we formulate evolutionary games dynamics based on the static game discussed 
in Section 13.11 We use generalized Smith-dynamics for channel selection and G-function based 
dynamics for rates. Combining them, we set up a framework of hybrid dynamics for the overall 
system. 

The action of each user has two components (aj,pj) G M_|_ x [0,1]'^. We use Pi as strategies 
that determine the fitness of user z's rate ai to receiver j. The rate selection evolves according to 
the channel selection strategy P. We may view channel selection as an inner game that involves a 
process on a short time scale but the rate selection is an outer game that represents the dynamical 
link via fitness on a longer time scale, [7], [8]. 

3.3.1 Learning the Weight Placed on Receiver 

Let a be a fixed rate on the capacity region. We assume that user i occasionally experiments 
the weights pij with alternative receivers, keeping the new strategy if and only if it leads to a 
strict increase in payoff. If the choice of receivers' weights of some users decreases the payoff or 
violates the constraints due to a strategy change by another user, he starts a random search for a 
new strategy, eventually settling on one with a probability that increases monotonically with its 
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realized payoff. For the above generating function based dynamics, the weight of switching from 
receiver j to receiver j' is given by 

'i]jj,{a,P) = max{0, Uij> {a, P) - Uij{a,P)f, 6 >1 

if the payoff obtained at receiver j' is greater the payoff obtained receiver j and the constraints 
are satisfied; otherwise, iljj'ip^ct) = 0. The frequencies of uses of each receiver is then seen as the 
selection strategy of receivers. 

The expected change at each receiver is the difference between the incoming flow and the 
outgoing flow. The dynamics is also called generalized Smith dynamics [2] and is given by 

Pvit) = E P^r m^ia^ P(i)) - p^At) E "^r (19) 

Let Xij{oi-,P{t)) := T,j'(zjPij'it)rifjia,'Pit))-Pij{t) ^j'^zj r]jj,{a,P{t)). Hence, the dynamics can 
be rewritten as pij = Xij(aiP(i))- For 6 = 1 the dynamics is known as Smith dynamics and has 
been used for describing the evolution of road traffic congestion in which the fitness is determined 
by the strategies chosen by all drivers. It has also been studied in the context of the resource 
selection in hybrid systems and migration constraint problem in wireless networks in [2]. 

Proposition 7. Any equilibrium of the game H with predetermined rates is a rest points of the 
generalized Smith dynamics \iy\}. 

Proof. The transition rate between receivers preserves the sign in the sense that, for every user, 
incoming flow from the receiver j' to j is positive if and only if the constraints are satisfied and 
the payoff to j exceeds the payoff to j' . Let q be a feasible point. We first remark that if the right 
hand side of ()19p is non-zero for some splitting strategy P, then 

d ■■= PijUij{a,P) = XijUij{a,P) 

= E P^^' ~ ^ijK"; P)) Vj'j 

= E (""^J ^"' ^) ~ P))] Vj'j 

which is strictly positive. Thus, if (a, P) is a Nash equilibrium then (a, P) satisfy the constraints, 
and Pij = 0, or r]jj,{a,P) = 0. This implies that (a,P) satisfies also ^("jP)) =0- D 

The following proposition says that the equilibria are exactly the rest point of (I19p . 

Proposition 8. Any rest point of the dynamics U9\) is a Nash equilibrium of the game H. 

The proof of Proposition [8] can be obtained by using Theorem III in |2j. Since the probability 
to switch from receiver j to j' is proportional to rjjj,, which preserves the sign of payoff difference, 
we can use the Theorems III in [2]. It follows that the dynamics generated by ry satisfy the Nash 
stationarity property. 

3.3.2 G-function Based Dynamics 

We introduce here the generating fitness function (G-function) based dynamics with projection 
onto the capacity region. The G-function approach has been successfully applied to non-linear 
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continuous games by Vincent and Brown [7], [8]. It is appropriate for our hybrid model because 
we can regard the channel selection as the variables in a fitness function. Users choose channel 
selection probabilities to aim at increasing their fitness of their rate choice. In our rate allocation 
game, to deal with constraints, we use projection into capacity region in order to preserve the 
trajectories feasible. Starting from a point in the polytope C, each user revises and updates its 
strategy according to a rate proportional to the gradient and its payoff subject to the capacity 
constraints. Let Gij be the fitness generating function of user i at receiver j defined on ^ 
satisfying 



t>NxJ 



Gij{v,a,P) 



Cj,Af - Pij/3ij{t) - E Pi'jPi'jit) \ , 

i'eAf\{i} 



if (a, P) satisfies in the hybrid capacity region. Notice that the term Cj^/^ — Yli'^iPi'jl^i'ji^) is 
maximum rate of i using channel j at time t. Hence, the G-function based dynamics is given by 



Pij/^ij- 



(20) 



with initial conditions /?ij(0) < {j}, where (3 = is defined in Proposition [5l which is of the 
same dimension as a, and ai{t) = ^j^j f3ij{t); ft is an appropriate parameter chosen for the rate 
of convergence. 

3.3.3 Hybrid Dynamics 

We now combine the two evolutionary game dynamics described in the previous subsections. Vari- 
ables a and P are both evolving in time. The overall dynamics are given by 




Ei'6jPi/(*)4,-(«W,P(0)-Kj(i)Ei'ej^i,'(a(t),P(t)) 
-fl Pij{t)Pij{t) - Cj^M + Yjii^iPi'j{t)Pi'j{t) Pij{t)l3ij{t) 



(21) 



All the equilibria of the hybrid evolutionary rate control and channel selection are rest point of the 
above hybrid dynamics. The following result can be obtained directly from Proposition [7] and (j20p . 



Proposition 9. Let (/3*,P*) be an interior rest points of the hybrid dynamics, i.e., (3*^ > 0, p*j > 
and x{c(* ,P) = 0. Then for all j, 

N f ^ \ 

i=i \j=i / 



3.4 Numerical Examples 

In this subsection, we illustrate the evolutionary dynamics in (j20p and (j2ip by examining a two-user 
and three-receiver communication system as depicted in Figuredl Let hn = 0.1, hi2 = 0.2, his = 0-3, 
for i = {1,2}. Each transmission power Pj is set to 1 mW for all i = 1, 2 and the noise level is set 
to cj2 = -20 dBm. 

In the first experiment, we assume that the rates of the users are predetermined to be a = 
[10, 20]^, the Smith dynamics in (j20p yield in Figure ([5]) and 1^ the response of pi and p2. It can 
be seen that the dynamics converge very fast within less than half a second. 
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Figure 4: Two users and three receivers 



In the second experiment, we assume that the probabihty matrix P has been optimally found 
by the users using (|19p . Figures [7] and [8] show that the /3 values converge to an equilibrium from 
which we can find the optimal value for a. Since these dynamics are much slower compared to 
Smith dynamics on P, our assumption of knowledge of optimal P for a slowly varying a becomes 
valid. 

In the next experiment, we simulate the hybrid dynamics in (j2ip . Let the probability pij of 
user i choosing transmitter j and the transmission rates be initialized as follows: 



P(0) 



0.2 0.3 0.5 
0.25 0.5 0.25 



a(0) 



0.2 
0.1 



We let the parameter Ji = 0.9. Figure [9] shows the evolution of the weights of user 1 on each of the 
receivers. The weights converge to be pij = 1/3 for all j within two seconds, leading to an unbiased 
choice among receivers. In Figure [TOl we show the evolution of the weights of the second user on 
each receiver. At the equilibrium, ps = [0.3484,0.4847,0.1669]'^. It appears that user 2 favors 
the second transmitter over the other ones. Since the utility Uij is of the same form, the optimal 
response set K* is naturally nonempty and contains all the receivers. Shown in Proposition [H the 
probability of choosing a receiver at the equilibrium is randomized among the three receivers and 
can be determined by the rates a chosen by the users. 

The /3-dynamics determines the evolution of a in (j2ip . In Figure [TH we see that the evolutionary 
dynamics yield a = [15.87,23.19]^ at the equilibrium. It is easy to verify that they satisfy the 
capacity constraints outlined in Section [2j It converges within 5 seconds and appears be much 
slower than in Figures [9] and [101 Hence, it can be seen that P— dynamics may be seen as the inner 
loop dynamics while /3— dynamics can be seen as an outer loop evolutionary dynamics. They evolve 
on two different time scales. In addition, thanks to Proposition [HI finding the rest points for the 
above dynamics ensures us finding the equilibrium. 



4 Concluding Remarks 

In this paper, we have studied an evolutionary multiple access channel game with a continuum 
action space and coupled rate constraints. We showed that the game has a continuum of strong 
equilibria which are 100% efficient in the rate optimization problem. We proposed the constrained 
Brown-von Neumann-Nash dynamics. Smith dynamics, and the replicator dynamics to study the 
stability of equilibria in the long run. In addition, we have introduced a hybrid multiple access 
game model and its corresponding evolutionary game-theoretic framework. We have analyzed the 
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Transmitter 1: Probabilities Time For Fisted aipiia=10 Transmitter 2:]i Value v.s. Time For Fixed aipha=20 




Figure 5: Transmitter 1: Probabilities Figure 6: Transmitter 2: Probabilities 
v.s. Time For Fixed qi=10 v.s. Time For Fixed a2=20 



Transmitter 1 : p Vaiue v.s. Time Transmitter 2: p Value v.s. Time 




Figure 7: Transmitter 1: (3 Value v.s. Figure 8: Transmitter 1: (3 Value v.s. 
Time Time 




Figure 9: Probability of Figure 10: Probability of 

Transmitter 1 Choosing Re- Transmitter 2 Choosing Re- Figure 11: Rates of Each 
ceivers ceivers Transmitter 

Nash equilibrium for the static game and suggested a system of evolutionary game dynamics based 
method to find it. It is found that the Smith dynamics for channel selections are a lot faster 
than the /3-dynamics, and the combined dynamics yield a rest point that corresponds to the Nash 
equilibrium. An interesting extension that we leave for future research is to introduce a dynamic 
channel characteristics: the gains hij{t) are time- dependent random variables. Another interesting 
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question is to find equilibria structure in the case of multiple access games with non-convex capacity 
regions. 
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